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Abstract 

We consider the general problem of finding the minimum weight b-matching on arbitrary graphs. We 
prove that, whenever the linear programming relaxation of the problem has no fractional solutions, then 
the cavity or belief propagation equations converge to the correct solution both for synchronous and 
asynchronous updating. 

Motivated by the cavity method , very fast distributed heuristic algorithms have recently been developed 
for the solution of random constraint satisfaction problems. In some cases, namely in the replica symmetric 
scheme, the algorithms generated by the cavity method are exactly of the form of a classic belief propagation 
(max-product or min-sum), i.e. a message-passing algorithm for efficiently computing marginal probabilities 
or finding the assignment with highest probability of a joint discrete probability distribution defined on a 
graph. The belief propagation (BP) algorithm converges to a correct solution if the associated graph is a 
tree, and may be also a good heuristic for some graphs with cycles. In other cases, e.g. when the space of 
solutions clusters into many subsets, the cavity method may lead to a more involved survey propagation (SP) 
algorithm [HE], in which some form of long range correlation among variables is included in the formalism. 

In this paper, we study the problem of finding the minimum weight b-matchings in arbitrary graphs via the 
min-sum version of the cavity/ BP algorithm. 

Let G ~ [V, E) be an undirected graph with edge weights Wij for each edge G E and node capacities 

bi for each node i £ V. The iterative message-passing algorithm based on synchronous BP for solving the 
weighted perfect b-matching problem is the following simple procedure: At each time, every vertex of the 
graph sends (real valued) messages to each of its neighbors. The message transmitted at time t from i 
to j is Wij minus the 6*^ minimum of the messages previously received by i at time t — 1 from all of its 
neighbors except j. At the end of each iteration, every vertex i selects bi of its adjacent edges that correspond 
to the bi smallest received messages. This procedure can be derived as the zero temperature limit of the 
cavity equations for the minimum b-matchning problem. In what follows we will use as synonymous cavity 
equation, belief propagation (BP) or min sum equations. 

We will show the following result: For arbitrary graphs G, and all sets of weights {wy}, after 0{n) iterations, 
the set of selected edges converges to the correct solution, i.e., to the minimum weight perfect b-matching 
of G, provided that the Linear Programming (LP) relaxation of the problem has no fractional solutions. 
Additionally we introduce a new construction, a generalized computation tree, which allows us to analyze the 
more complicated case of BP with an asynchronous updating scheme, and prove convergence and correctness 
of it when each edge of the graph transmits at least 6{n) messages. 

Our proof gives new insight of the often-noted but poorly understood connection between the cavity method 
(or BP) and LP through the dual of the LP relaxation. 

We also modify our BP algorithm and its analysis to include the problem of finding the non-perfect weighted 
b-matchings. Independent work on this aspect can be found in [3]. Previous exact results on bipartite 
graphs was obtained in [1] for b = 1 and then was extended to all b in [S]. 
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The weighted b-matching problem is an important problem in combinatorial optimization. For extensive 
surveys see j6] and [7]. In physics, the study of the random 1-matching problem by the cavity method 
goes back to the work of Mczard and Parisi [5] who made a celebrated conjecture for the expected optimum 
weight (7r^/6) that was proven to be exact a decade later by Aldous [S]- BP algorithms have been the subject 
of extensive study in several communities (see [3] and [20j for a detailed survey of rigorous results about 
BP). Recent works have also suggested a connection between the BP algorithm and linear programming 
(LP) in particular problems. A relationship between iterative decoding of channel codes and LP decoding 
was studied in [TU] , [Tl] , [T^] . Other relationships were noted in the context of BP algorithms with convex 
free energies [T3], [13], and in the case of BP algorithms for resource allocations [TS]. For weighted 1- 
matchings, the connection was studied [16| in the context of similarities between BP equations and the 
primal-dual auction algorithm of Bertsekas [T7]. Finally, we note that the BP equations for solving the 
weighted matching problem which we use in this paper have been previously studied in [16| . [5]. These 
equations are also similar (though not identical) to the cavity equations for weighted matching problems and 
traveling salesman problems which are found in the statistical physics literature, see e.g. [8], [9], [l8j, and 

m- 

The main contributions of our results can be summarized as follows: 

1. The properties of the BP equations for the weighted matching was first used in and its correctness 
and convergence was shown for bipartite graphs with unique optimum solution. The same technique 
was used in [S] to extend the result to b-matchings in bipartite graphs. But this technique fails for 
graphs containing cycles with odd length. In order to bypass this difficulty we use a completely different 
tool, complementary slackness conditions of the LP relaxation and its dual, which is independent of 
the graph structure. 

2. The connection between LP and the cavity method (or BP) is particularly important given that LP 
is a widely used technique for optimization. We show both convergence and correctness of the BP 
algorithm when LP relaxation has no fractional solutions, thus establishing a solid link between the 
two methods. In some recent work, |14| . which generalizes methods of |13| . the connection of the 
BP algorithm and LP relaxation is studied in the converged case of the BP. The authors also study 
interesting variations of the BP which have convex free energies. 

3. The asynchronous BP, which includes the synchronous version as a special case, has been a more 
popular version for practical purposes. But, due to its more complicated structure, it has not been the 
subject of much rigorous study. Here we provide correctness and convergence proof of asynchronous 
BP for a combinatorial optimization problem based on the construction of a generalized computation 
tree, which can be used for the analysis of the both convergence and correctness of asynchronous 
message-passing algorithms as the cavity method. 

Consider an undirected simple graph G — {V, E), with vertices V — {1, . . . ,n}, and edges E. Let each edge 
have weight Wij G M. Denote the set of neighbors of each vertex i in G by N{i). Let b = (6i, . . . , 5„) 
be a sequence of positive integers such that bi < degc{i)- A subgraph M of G is called a h-matching (perfect 
h-matching) if the degree of each vertex i in M is at most bi (equal to bi). Denote the set of b-matchings 
(perfect b-matchings) of G by Mcih) (PMG(b)), and assume that it is non-empty. Clearly PMG(b) C MG(b). 

The weight of a (perfect or non-perfect) b-matching M, denoted by Wm, is defined by Wm = X]y l{i,i}eA/- 
In the next two sections, we will restrict ourselves to the case of perfect b-matchings. The analysis is ex- 
tended to (possibly non-perfect) b-matchings in [20] • The minimum weight perfect b-Matching (b-MWPM), 
M* , is defined by M* = argmin^gp]y[^(^,-) Wm- The goal of this paper is to find M* via a min-sum belief 
propagation algorithm. Throughout the paper, we will assume that M* is unique. 

Linear Programming Relaxation. Assigning variables Xij £ {0,1} to the edges in E, we can express 
the weighted perfect b-matching problem as the problem of finding a vector x e {0, l}'^' that minimizes the 
total weight X]ije_E ^ij^ij^ subject to the constraints X]jejv(i) '^u ^ ^' ^'^^ i (z V. Relaxing the constraint 
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that Xij is integer, this leads to the following linear program and its dual: 



mm 
subject to 



Primal LP 



max 
subject to 



Ay > 



Dual LP 



W{i,j}eE 



(1) 

We say the LP relaxation ([T|) has no fractional solution if, every optimal solution x of LP satisfies x G 
{0, 1}I^L Note that absence of fractional solutions implies uniqueness of integer solutions, since any convex 
combination of two integer solutions is a solution to the LP as well. We want to show that the BP algorithm 
for our problem converges to the correct solution, provided the LP relaxation ([T|) has no fractional solution. 

Complementary Slackness Conditions. Complementary slackness for the LP and its dual state that the 
variables x* = {x*j) and y* = (j/*). A* = (A*^) are optimum solutions to the LP relaxation and its dual ([T]), 
respectively, if and only if for all edges {i,j} of G both x*j{wij + X*j - y* - y*) = 0. and (x*, - l)A*j = hold. 
Using the fact that the LP has no fractional solution, one can deduce the following modified complementary 



slackness conditions: For all {i,j} € M*\ Wij + A* 



y* and for all {i,j} ^ M*; A* 



0. 



By these conditions and the fact that X*j > 0, we have that Wij < y* + y* for all {i,j} € M*, and 
Wij > y* + y* for all {i,j} ^ M*. However, as the counterexample given in [20] shows, it is in general 
not true that these inequalities are strict even when the LP has no fractional solution. Let S be the set 



of those edges in G for which 



y*\ > 0. We will assume the minimum gap is e. i.e. e 



y*\ > 0. Throughout this paper we assume that there exist an edge in G for which 



the strict inequality \wij — yi — y]] > holds and therefore e > is well defined. The other cases, where for 
each {i,j} S E the equality Wij = y* +yj holds, happens only for special cases are discussed in [20j and can 
be treated similarly. Let also 

(2) 



L = max \y* 

l<i<n 



Algorithm and Main Result. The following algorithm is a synchronous implementation of BP for finding 
the minimum weight perfect b-matching (b-MWPM). The main intuition behind this algorithm (and, indeed, 
all BP algorithms) is that each vertex of the graph assumes the graph has no cycles, and makes the best 
(greedy) decision based on this assumption. Before applying the BP algorithm, we remove all trivial vertices 
from the graph. A vertex i is called trivial if degaii) = hi. This is because all of the edges adjacent to i 
should be in every perfect b-matching. Therefore the graph can be simplified by removal of all trivial vertices 
and their adjacent edges. 



Algorithm Sync-BP. 



(1) At times t = 0, 1, . . ., each vertex sends real-valued messages to each of its neighbors. The message of 
i to j at time t is denoted by rrii^j (t) . 

(2) Messages are initialized by mi^j{Q) = Wij for all {i,j} G E (the messages can actually be initialized 
by any arbitrary values [20] ) 

(3) For t > 1, messages in iteration t are obtained from messages in iteration t — 1 recursively as follows: 



y{t,j}eE: 



iit) = - 6- '-min^gjv(»)\{i} 



rui^iit - 1) 



(3) 



where fc* -min[A] denotes the k minimu of set A. 



^Note that the t'*''-min^gjv(i)^{j-j is well defined since we assumed that all trivial vertices are removed and thus there are at 
least bi + I elements in the set N{i) for each i. 
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(4) The estimated b-MWPM at the end of iteration t is M(t) = W^^^Eiit) where Ei{t) = {{i, ji}, ■ ■ ■ , {h jb,}} 
is such that N{i) = {ji, j2, . • -Jdego^i)} and mj^^i{t) < mj^^i{t) ■■ < mj^^^^^^^^^{t). i.e., among all 
I's neighbors, choose edges to the bi neighbors that transfer the smallest incoming messages to i. 

(5) Repeat (3)-(4) until M{t) convergefl 



In Lemma [U we will show the main intuition behind the equation ([3]) and how it is derived. But wc note 
that one can also use the graphical model representations of [4] , [5] , [21] to obtain the standard BP equations 
for this problem, which, after some algebraic calculations, yield the recursive equation ([3]). 

The main result of the paper says that the above algorithm, which is designed for graphs with no cycle (i.e., 
for trees), works correctly for a much larger family of graphs including those with many short cycles. 

Theorem 1 Assume that the LP relaxation (Qp has no fractional solution. Then the algorithm Sync-BP 
converges to M* after at most \^^~\ iterations. 

If the LP relaxation ^ has a fractional solution whose cost is strictly less than Wm* , then [5T] , [5] have 
shown for the case of 1-matching that BP does not converge to M* . It is straightforward to generalize this 
to perfect b-matching as well. But for the case in which the LP relaxation has a fractional solution whose 
cost is equal to Wm* , BP fails in general. This is because the bf^ minimum in equation ([3]) is not unique, 
and one needs an oracle to make the right decision. If such an oracle exists, then BP converges to M*. 

In what follows we first display the connection between the Sync-BP equations and the so-called computation 
tree. Next we discuss the how the complementary slackness conditions is related to alternating paths in the 
graph G. These results arc eventually used to prove that, when the LP relaxation has no fractional solutions, 
then solutions on the computation tree are the same as the solutions on the original graph G. 

Analysis of the Synchronous BP via Computation Tree. The main idea behind the algorithm Sync- 
BP is that it assumes the graph G has no cycle. In other words, it finds the b-MWPM of a graph G' that has 
the same local structure as G but no cycles. The precise definition of the computational tree for Sync-BP 
goes as follows. 

Computation Tree. For any i £ V, let Tf be the t-\eve\ computation tree corresponding to i, defined as 
follows: Tf is a weighted tree of height t + 1, rooted at i. All tree-nodes have labels from the set {1, . . . ,n} 
according to the following recursive rules: (a) The root has label i, (b) The set of labels of the degdi) 
children of the root is equal to N{i), and (c) If s is a non-leaf node whose parent has label r, then the set 
of labels of its children is A(s)\{r}. T!* is often called the unwrapped tree at node i. The computation tree 
is well known technique for analyzing algorithms and constructed by replicating the local connectivity of 
the original graph. The messages received by node i in the belief propagation algorithm after t iterations in 
graph G are equivalent to those that would have been received by the root i in the computation tree, if the 
messages were passed up along the tree from the leaves to the root. A subtree of edges in the computation 
tree is called a perfect tree-h-matching if for each non-leaf vertex with label i we have degM{i) = ^i- Now 
denote the minimum weight perfect tree-b-matching (b-TMWPM) of the computation tree T/ by M*{Tl). 
The following lemma shows that Sync-BP can be seen as a dynamic programming procedure that finds the 
minimum weight perfect tree-b-matching over the computation tree. Figure [T] shows a graph G and one of 
its corresponding computation tree. 

Lemma 1 The algorithm Sync-BP solves the h-TMWPM problem on the computation tree. In particular, 
for each vertex i of G, the set of Ei it) which was chosen at the end of iteration t by Sync-BP is exactly the 
set of bi edges which are attached to the root in h-TMWPM of Tf . 

Lemma [1] characterizes the estimated b-MWPM, M{t), and will be used later in the proof of the main result. 

" The subgraph M{t) is not necessarily a perfect b-matching of G but wc will show that after 0(n) iterations it will be the 
minimum weight perfect b-matching. 
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(a) (b) 



Figure 1: Part (a) shows a graph G where dashed and gray edges represent a 1-matching. Part (b) shows 
the computation tree T| corresponding to G where the set of dashed and gray edges form a 1-TMWPM. 

The foUowing technical Lemma is also crucial for the proof of equivalence between BP and LP. It connects 
the complementary slackness conditions to paths on the graph G and on the computation tree. This lemma 
provides the connection between the absence of fractional solutions and the correctness of BP. 

Definition A path P ~ (ii, ■ ■ • , *fc) in G is called alternating path if it has the following two properties: 
(i) There exist a partition of edges of P into two sets A, B such that either [A C M* , B n M* = 0) or 
(A n M* = , Be M*). Moreover A (B) consists of all odd (even) edges; i.e. A {(11,12), (i3,u), ■ • ■} 
(B = {(12,13), (i4,*5), ■ • ■}), and (ii) The path P might intersect itself or even repeat its own edges but no 
edge is repeated immediately. That is, for any l<r</s — 2: ir ^ ir+i and i^. ^ ir+2- P is called an 
alternating cycle if ii = ik- 

Lemma 2 Assume that the LP relaxation (0]) has no fractional solution. Then for any alternating path P 
of length at least 2n, there exists an edge {i,j} G P such that the inequality \wij — y* — Vjl > holds. That 
IS, 0. 

Proof of Theorem [T] We will prove Theorem [U namely that if the LP relaxation ([T]) has no fractional 
solution and hence M* is unique, then Sync-BP converges to the correct b-MWPM. We will do this by 
showing that if the depth of computation tree is large enough, then for any vertex i, its neighbors in Af * 
(b-MWPM of G) are exactly those children that are selected in Af*{Tl) (b-TMWPM of T^). Here is the 
main lemma that summarizes the above claim: 

Lemma 3 If the LP relaxation [J^ has no fractional solution, then for any vertex i of G and for any t > 

the set of edges that are adjacent to root i in J\f*(Tl) are exactly those edges that are connected to i in M* . 

The proof of Lemma [3] is the main technical part of this work. The high level overview of the underlying 
argument goes as follows. Consider the computation tree (T/) rooted at vertex i and look at 7V*(T!/). We 
will assume that the claim of the lemma does not hold. That is, we assume that at the root, Af*{Tl) does 
not choose the same edges as M*-edgcs adjacent to i. Then we use the property of perfect tree-b-matchings, 
namely that each non-leaf vertex j is connected to exactly bj of its neighbors, to construct a new perfect 
trce-b-matching on the computation tree. This new perfect tree-b-matching is going to have less total weight 
if the depth of the computation tree is large enough. This last step uses an alternating path argument which 
is a highly non-trivial generalization of the technique of [4] for the case of perfect 1-matching in bipartite 
graphs. For this part we will use the solutions to the dual LP ([1]). 

Proof of Lemma [3] Let us denote the lifting of a perfect b-matching M* to a perfect tree-b-matching on 
Tj* by AA* . That is, M.* consists of all edge of the computation tree with endpoint labels i,j such that 
{i,j} S M* as an edge in G. The goal is to show that J\f*{Tl) and M* have the same set of edges at the 
root of the computation tree. To lighten the notation, we denote the b-TMWPM of T/ by J\f* . 

Assume the contrary, that there exist children i-i,ii of root i such that {i,ii} G M-*\J\f* and {z,i_i} £ 
Af*\M*. Since both M* , JV* are perfect tree-b-matchings, they have bi^ edges connected to ii. Therefore 
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there exist a child i2 of ii such that {11,12} G N'*\M* . Similarly there is a child i-2 of z_i such that 
{i_i,i_2} £ A^*\A/'*. Therefore we can construct a set of alternating paths Pi, £ > 0, in the computation 
tree, that contain edges from A4* and TV* alternatively defined as follows. Let io ~ root i and Pq = (io) 
be a single vertex path. Let Pi = (i_i,io,«i), P2 = (i-2, i-i, ioi iii ^2) and similarly for r > 1, define 
P2r+i and P2r-+2 recursively as follows: P2r+i = (i_(2r-+i), ■f2r, »2r+i), -P2r+2 = («-(2r+2),-f2r+i,»2r+2) where 
i_(2r-+i), *2r+i are nodes at level 2r + 1 such that {i2r,*2r+i} £ M*\M* and {i-2r, «-(2r+i)} G A/'*\A^*. 
Similarly j_(2r+2)j *2r+2 are nodes at level 2r+2 such that {i2r+i, *2r+2} G A/'*\A^* and {i-(2r+i)j *-(2r+2)} G 
A^*\A/'*. Note that, by definition, such paths for < ^ < < exist since the tree T/ has t + 1 levels and 
can support a path of length at most 2t as defined above. Now consider the path Pt of length 2t. It is an 
alternating path on the computation tree with edges from M* and Af* . Let us refer to the edges of M* 
(TV*) as the A^*-edges (TV*-edges) of Pt. We will now modify the perfect tree-b-matching TV* by replacing 
all TV*-edges of Pt with their complement in Pt (TW*-edges of Pt). It is straightforward that this process 
produces a new perfect tree-b-matching A/"' in T/. 

Let us assume, for the moment, the following lemma: 

Lemma 4 The weight of the perfect tree-h - matching M' is strictly less than that of Af* on Tf . 

This completes the proof of Lemma |3] since Lemma 2] shows that TV* is not the minimum weight perfect 
tree-b-matching on T/, leading to a contradiction. I 



Proof of Lemma [4] It suffices to show that the total weight of the TV*-edges of Pt is more than the total 
weight of T\4*-edgcs of Pt. For each vertex v G Pt consider the value y* from the optimum solution to 
the dual LP H]). Using the inequality Wij < y* + y* for edges of M* , we obtain: j}ePtnM' — 

{j2l-=-t Vir^ ~ J ~ where ki is the number of A^*-cdges of Pt that belong to S, i.e., the number of 
T\4*-edges of Pt endowed with the strict inequality Wij < y* + yj , with a gap of at least e. On the other hand, 

using the inequality > yf-Fy* for edges of TV* we have: J2{i,J}(^p^n^f'■ '^H ^ (Et=-i "^.^t+i, +^=26 
where now k2 is number of A/'*-edges of Pt that belong to S, or equivalently the number of times the inequality 
Wij > y* + y*j is strict with a gap of at least e. One finds 

x-^ (^) (c) 



Here (a) uses definition of L from eq. [Hand (6) uses the fact that for all i,j: A*,- > 0. The main step is (c), 
which uses Lemma [2] as follows. Path Pt has length 2t, and each continuous piece of it with length 2n has a 
projection to the graph G which satisfies the conditions of Lemma [2j This means the path has at least one 
edge from the set S. Thus (fci + ^2) > |^ > This completes the proof of Lemma |4l I 



Extension to Possibly Non-Perfect b-Matchings. Here we note that the algorithm and the results 
of the previous sections can be easily generalized to the case of b-matchings (subgraphs H oi G such that 
degree of each vertex i in iJ is at most hi). Let U{H) C be the set of unsaturated vertices of G (vertices 
i € V such that degnii) < h). The minimum weight b-Matching (b-MWM), H* , is the b-Matching such 
that H* = argmaxj^gjyj^ (-jj-) Wh. Note that H* does not include any edge with positive weight because 
removing such edges from H* reduces its weight while keeping it a b-matching. 

Asynchronous BP. In the remaining we study the asynchronous version of the BP algorithm. The update 
equations are exactly analogous to the synchronous version, but at each time only a subset of the edges are 
updated in an arbitrary order. Consider the set E of all directed edges in the G; i.e., E ^ {{i ^ j) s.t. i ^ 
j G V}. Let A be a sequence £'(l),i5(2), ... of subsets of the set E. Then the asynchronous BP algorithm 
corresponding to the sequence A can be obtained by modifying only the update rule in the step (3) of the 

algorithm Sync-BP as follows: mi^j{t) ~ Wij — 6*'*-min^gjv(i)\{i} m-e^iit ~ 1) 



if {i j) e E{t) and if 



{i j) ^ E{t) then the message will not be updated, i.e. it remains equal to mi^j{t — 1). 
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This is the most general form of the asynchronous BP and it includes the synchronous version {E{t) = E 
for all i = 1, 2, . . .) as a special case. In many applications, a special case of the asynchronous BP is used for 
which each set E{t) consists of a single element. 

We assume that the sequence A of the updates does not have redundancies. That is, no edge direction 
[i j) <E E is re-updated before at least one of its incoming edge directions {{£ — > i) for £ g is 
updated. More formally, if (i j) £ E{t) n E{t + s) and (i j) ^ yJlZ^E{t + r), then at least for one 
£ £ N{i)\{j}, we should have {£ ^ i) £ U"z}£;(t + r). 

Let us denote the above algorithm by Async-BP. Wc claim that, if each edge direction {i ^ j) £ E is 
updated 9{n) times, then the same result as Theorem [T] can be proved here. That is, let u{t) be the 
minimum number of times that an edge direction of the graph G appears in the sequence . . . ,E{t); 



i.e., u{t) = min^-_ 



{£ : s.t. 1 < e <t and {i ^ j) £ E{£)} 



non-decreasing function of t. We claim that the following result holds: 



From the definition, u{t) is a 



Theorem 2 Assume that the LP relaxation (Qp has no fractional solution. Then the algorithm Async-BP 
converges to M* after at most t iterations, provided u{t) > "^uLl , 

Proof of the above theorem relies on the notion of generalized computation tree for the asynchronous version 
of the BP algorithm which will be given in the longer version of this paper [20j . 

Finally we note that the same algorithm as Async-BP and the same result as Theorem [5] can be stated and 
proved for the (possibly non-perfect) b-matchings as well. 
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